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Article Info ABSTRACT 

Article history: This study presents characterization of cracking in pavement distress using 
image processing techniques and k-nearest neighbour (KNN) classifier. 

Received Sep 28, 2018 The proposed semi-automated detection system for characterization on 

Revised Nov 25, 2018 pavement distress anticipated to minimize the human supervision from 

Accepted Dec 7, 2018 traditional surveys and reduces cost of maintenance of pavement distress. 


The system consists of 4 stages which are image acquisition, 
image processing, feature extraction and classification. Firstly, a tool for 
Keywords: image acquisition, consisting of digital camera, camera holder and tripod, 
is developed to capture images of pavement distress. Secondly, 
image processing techniques such as image thresholding, median filter, 
image erosion and image filling are applied. Thirdly, two features that 
k-nearest neighbour represent the length of pavement cracking in x and y coordinate system 
Pavement distress analysis namely delta_x and delta_y are computed. Finally, the computed features is 
fed to a KNN classifier to build its committee and further used to classify the 
pavement cracking into two types; transverse and longitudinal cracking. 
The performance of kNN classifier in classifying the type of pavement 
cracking 1s also compared with a modified version of KNN called fuzzy kNN 
classifier. Based on the results from images analysis, the semi-automated 
image processing system is able to consistently characterize the crack pattern 
with accuracy up to 90%. The comparison of analysed data with field data 
shows good agreement in the pavement distress characterization. Thus the 
encouraging results of semi-automated image analysis system will be useful 
for developing a more efficient road maintenance process. 
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1, INTRODUCTION 

Pavement distress is a condition of pavement that shown imperfection on the surface of pavement. 
Cracking is one of the most common and important types of asphalt pavement distress [1-2]. 
Generally, cracking distress can be divided into three main types which are longitudinal, alligator and 
transverse cracking [3-5]. Pavement maintenance is generally identified as work accomplished to preserve or 
extend the pavement’s service life until major rehabilitation or complete reconstruction is performed. 
It is classified by function as either routine or preventive. In Malaysia, the pavement distress analysis is 
normally conducted manually using visual inspection by road maintenance operator. The manual system is 
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costly, hazardous to the operator and expensive due to the longer hours of assessment by operator [6]. 
Therefore there is a need to develop a more efficient automated system to perform pavement distress 
analysis. Hence the application of artificial intelligence (AI) was explored to assist in characterization of 
pavement distress. AI system needs very minimal labour to verify the cracking pavement distress that allows 
faster decision to be made in selecting suitable treatment method. 

From the previous works, most of existing methods of AI for identifying defects on pavement 
surface are based on characteristics of crack. They consider that crack pixels are darker than the ones that 
surround them. Different researcher used fixed threshold to detect dark pixels (cracks). As a key approach of 
crack detection, thresholding can affect the efficiency of image segmentation during the image 
processing [7]. Although there are many methods to compensate the crack and remove the noise, a good 
thresholding method is needed to obtain a clear binary image. A several image enhancement techniques, 
smoothing, low-pass filtering, sharpening were compared by research of Rababaah [7 and the best result was 
using the Median filter after thresholding. Median filtering can be used to reduce the noise while preserving 
much of the details in the image. Essentially, there are a few ways to controlled condition of an image at 
image acquisition stage. For example, [7] applied the technique on pavement survey vehicles by modified the 
back of the Mobile Laboratory with attached a digital camera, as depicted in Figure |. It permits high speed 
surveys of geo-referenced digital images of the pavement to be made by integrating the GPS and the digital 
image data acquisition system. 





Figure 1. Pavement survey vehicle [6] 


Another method to extract binary image is to use an adaptive fuzzy thresholding where it can 
identify pavement distress from original images. Saar and Talvik [8] found the main idea behind this method 
is based on the fact that crack pixels in pavement images are both continuous and darker than their 
surroundings. Figure 2 shows an example of threshold process where at the left is an original image and at 
the right 1s image after threshold process. 





a 


Figure 2. Original (left) and threshold (right) image [7] 


Most recent, Banharnsakun [6] combines artificial bee colony (ABC) algorithm and artificial neural 
network (ANN) called “ABC-ANN” for crack type classification. Thresholding method based on ABC 
algorithm was initially used to partition captured images into two regions, distressed and non-distressed 
regions based on a thresholding method. Then, three features namely vertical distress measure, horizontal 
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distress measure and total number of distress pixels, are computed from the distressed region. Finally, 
an ANN is used to classify these input features into three types, transversal crack, longitudinal crack, 
and pothole. By using the proposed method, the author claimed to achieve 20 % increase compared to the 
existing algorithm. 

This study proposed to use k-nearest neighbor (kKNN) and fuzzy k-nearest neighbor (fuzzy kKNN) to 
classify crack types. Both classifiers were selected due to very simple, require less data and reported to have 
highly efficient and effective in solving classification problems [9-11]. 


2. RESEARCH METHOD 
The research methodology consists of four (4) parts namely image acquisition, image processing, 
feature extraction and classification. Figure 3 shows the flowchart methodology of this study. 


2.1. Image Acquisition 

In this process, pictures were taken by using a digital camera during daylight with its optical 
perpendicular to the road surface and its lateral edge parallel to the axis. Image of pavement distress was 
collected from surrounding area of Ui1TM Cawangan Pulau Pinang and arbitrary streets in Pulau Pinang 
which will be used for this project. To get a good result, the image must be clear image with cracks without 
any obstacle such as shadow, sand inside the crack or any unwanted things that will disturb the image 
processing. Referring to [7], to keep the picture maintain consistent, reliable and systematic, some precaution 


must be considered during image acquisition: 


Image acquisition 
Image processing 


Feature extraction 


Classification sing KNN 
and fuzzy KNW classifier 





End 


Figure 3. Flowchart of the proposed research methodology 


a) Illumination: For better quality, image capturing is conducted during a bright day and at similar timing 
for consistency. Image brightness and contrast will affect the process reading to detect type of crack. 
Therefore, illumination is an important to ensure the image will captured with a good quality for image 
processing data. 

b) Distance from the camera to the pavement: The distance is fixed at the same level during taking a 
picture of pavement distress. It will affect the size of crack and the relative gabs between crack object to 
detect and classify the type of crack. 

c) Expert Validation: Get a consulted and approved from the expert to develop a correct way of data 
collection technique to get an accurate result. 

Based on the abovementioned precautions, the tool has been built as shown in Figure 4. In order to 
obtain consistent, reliable and efficient images collection, the digital camera is mounted on a camera holder 
with consistent height distance of 80cm from the pavement surface. The image capturing process was 
performed during daylight at similar timing from 8.00 am to 11.00 am. For a better result, data collection at 
noon starting from 12.00 pm until 3.00 pm was avoided to minimized shadow interruption. 
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Figure 4. Tool for image acquisition 


2.2. Image Processing 

Image processing 1s an important stage for this application system with several processes that must 
be through to get a result. A simple image processing was unable to give an accurate reading to the image 
that we want to process due to the size of image is too large, non-uniform lighting or weather conditions or 
the contrast of image 1s in low quality. Just because of this problem crack that presence on the road surface 
cannot be detect or classify. Hence, image processing will be used to enhance the image so that it can be 
process based on what we want for the result. The image processing techniques was implemented using 
MATLAB Image Processing Toolbox. 

The proposed image processing steps in this study is described as a process that involved RGB to 
grayscale image conversion, image segmentation, median filter, image erosion, image filled and image 
remove small area. The analysis of image was start with uploading images to MATLAB program. 
Next, the RGB to grayscale image conversion was applied to reduce computational quantity and complexity 
during thresholding. 

Image segmentation means process to divide the image into object and background. In this study 
manual image thresholding was selected due to the problems of images with different condition and 
illuminations. The process is achieved by gradually increase the threshold value, t from zero to the value that 
can visually eliminate most of the background and retaining most of the object (pavement crack) in the 
image. Then, the median filter takes place which means process of reduces of noise of image. Subsequent, 
image erosion was implemented to the resultant image and image filled was used to fill a holes on object of 
image to get a sharp of object image. Lastly, the process of remove unwanted image using image remove 
small area and that is the resultant image for image processing stage. 


2.3. Feature Extraction 

This process 1s considered as process of converting the given data into classifier acceptable format 
and for the classification task are computed for the normalized images. The image 1s to be represented into 
relevant features for recognition process, so that the classifier 1s applied to recognize the object. This feature 
is useful in finding the length of major axis and minor axis by propose delta_x and delta_y to classify type of 
crack. If the size of delta_x is greater than delta_y it is means the type of crack pavement is transverse crack. 
Other than that, if the size of delta_y 1s greater than delta_x it is means the type of crack pavement is 
longitudinal crack. Figure 5 shows the MATLAB source code to calculate delta_x and delta_y. The program 
starts by finding all the pixels coordinates that representing the object (the white pixel) in BW image. 
Then, the maximum and minimum coordinate of y-axis and x-axis is determined using max(_) and min( ) 
functions. Finally the delta_x and delta_y are determined by finding the difference between the maximum 
and minimum coordinates. 


[y, x] = find(BW); 
xl =min(x); x2 = max(x); 


yl=min(y); y2 = max(y); 
delta_x = x2 - xl; 
delta_y = y2 - yl; 





Figure 5. MATLAB source code to determine delta_x and delta_y 
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2.4. Crack Classification using KNN and Fuzzy KNN Classifier 

The last part is classification stage which is performed using kKNN and fuzzy kNN classifiers. 
The KNN classifier is one of the classifier that are very simple to understand but work incredibly well in 
practice and become well known nowadays in the pattern of classification procedure. This method has been 
used in many applications in the field of data removal, numerical pattern recognition and many others. 
The kKNN classifier is widely use in the pattern recognition community due to its good performance in 
practical application. KNN classifies an object based on closest training examples by a majority vote of 
its neighbour. 

The fuzzy kNN classifier is a modified version of kKNN classifier. The fuzzy k-nearest neighbour 
algorithm assigns class membership to a sample vector rather than assigning the vector to a particular class as 
in the kKNN [13]. No arbitrary assignment is the advantages are made by the algorithm. In addition, a level of 
assurance to accompany the resultant classification had been provided from the vector’s membership values. 
In both methods, the number of nearest neighbor, k is usually set to odd values (3, 5, 7, 9, etc.) to ensure there 
is no equal number of example is selected. 

In this study, the kKNN and fuzzy kNN target (cracking type) is assigned 1 as longitudinal crack 
and 2 as transverse crack. Example of longitudinal and transverse crack is illustrated in Figure 6. Both types 
are the most common pavement distress type encountered during road survey. Meanwhile, the number of 
nearest neighbor & is set similar to 3 for both KNN and fuzzy kNN classifiers. The value was determined 
experimentally and found to be the sufficient in classifying the pavement cracking types. 
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(a) Transverse crack ) Longitunidal crack 


Figure 6. Types of crack classified using KNN and fuzzy kNN algorithm 
3. RESULTS AND ANALYSIS 


During image acquisition step, a total of 150 images (75 transverse crack and 75 longitunidal crack) 
were captured in different areas. Example of the captured images are shown in Figure 7. 





Figure 7. Examples pavement crack images capture using the image acquisition system. 


It can clearly be seen that the pavement crack pixels are usually darker than the surrounding pixels. 
However, the surrounding pixels were found to have different intensities in different images. This is due to 
inconsistency in lighting conditions and non-uniform pavement surface. The problems will result in difficulty 
in achieving satisfactory image segmentation results and further lead to misclassification of pavement cracks. 

The study proposed to use manual thresholding to overcome the problem. The manual thresholding 
is performed by gradually increase the threshold value, t from zero to the value that can visually eliminate 
most of the background and retaining most of the object (pavement crack) in the image. Figure 8(a) gives an 
example of pavement crack image and Figure 8(b) is the resultant image after applying the manual threshold 
(t=0.29). Table 1 gives the minimum, maximum and average threshold value selected during the manual 
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thresholding process. These value were obtained thorough analysis of 150 images, as mentioned previously. 
Table 1. The Minimum, Maximum and Average Threshold Value for 150 Crack Images 


Threshold value, t 


Type of crack Minimum Maximum Average 
Longitunidal 0.25 0.39 0.35 
Tranverse 0.26 0.38 0.34 
Average 0.35 


It can be seen that the manual image thresholding has partially removed the background pixels with 
higher intensity values. Unfortunately, the method was unable to eliminate the background pixels having 
intensity similar or lower than the pavement crack intensities. Therefore, a median filter was applied to 
reduce the image noise. Subsequent, image erosion was implemented to the resultant image and image filled 


was used to fill a holes on object of image to get a sharp of object image. Lastly, the process of remove 
unwanted image using image remove small area and that is the resultant image for image processing stage. 
Figure 8 give an example of a crack image and the results after applying the image processing techniques. 
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(a) Original image (b) RGB to grayscale image conversion 
and manual threshold (t=0.29) 





(c) Median filter | (d) Image erosion 
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(e) Filled image (f) Remove small area image 


Figure 8. Example of pavement cracking image and its resultant images after applying image 
processing techniques 


Feature extraction was applied to the processed images to extract pavement crack information. 
Two features called the length of major axis and minor axis by propose delta_x and delta_y were extracted. 
Both features were then fed into KNN classifier to identify the type of crack. Table 2 gives examples of crack, 
value of delta_x and delta_y and type of crack classified by the kKNN classifier. 

For each crack type, 25 randomly selected images was used to evaluate the classification 
performance of kNN classifier while the remaining images were used to build the kKNN committee. 
In this study, the number of nearest neighbor, k is set to 3 for all analyses. Classification performance of KNN 
classifier was evaluated in term of accuracy and processing time, and further compared with the fuzzy kNN 
classifier. Processing time refers to the time interval between the image features extraction and classification 
step. The image processing step was not accounted for processing time due its manual processing. 
Table 3 summarises the achievement of the classifiers. For a fair comparison, the number of nearest neighbor 
k for fuzzy KNN classifier is set to 3, similar to the kKNN classifier. 
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Referring to Table 3, classification accuracy of kKNN classifier was found similar to the fuzzy kKNN 
classifier. Three (3) logitunidal and two (2) tranverse crack images was wrongly classified by both classifier. 
Further investigation on the resultant image found that the presence of noise after the image processing step 
has led to misclassification of the crack type. However, classification using kKNN classifier is more efficient 
since its processing time is slightly lower than the fuzzy kNN classifier, and thus chosen as the 


final classifier. 


Table 2. Examples of Pavement Crack, Value of delta_x and delta_y and Type of Crack Classified 


by KNN Classifier 
Field data Classification using KNN Remark 





















poe Identical 
Type: Transverse crack Delta x: 2360 pixels 
Length: 68 cm Delta y: 424 pixels 
Result kKNN:2 
? 
; 
‘d Identical 
; 
Type: ; Delta x: 300 pixels 
Longitudinal crack Delta y: 1262 pixels 
Length: 98 cm Result KNN:1 
Identical 
% 
L, 
Type: Longitudinal crack Delta x:126 pixels 
Length: 89 cm Delta y: 1260 pixels 
Result KNN:1 
Fee pares ee Identical 
Type: Longitudinal crack Delta x: 441 pixels 
Length: 101 cm Delta y: 3257 pixels 
Result kKNN: 1 
. i Identical 
Type: Transverse crack Delta x: 2853 pixels 
Length: 72 cm Delta y: 487 pixels 


Result kNN:2 


Table 3. Classification Performance using kKNN and Fuzzy kNN Classifier for 
Longitudal and Tranverse Crack 


kNN FkKNN 
Classiffication accuracy Average processing Classiffication Acerage processing time 
Type of crack (%) time (s) accuracy (%) (s) 
Longitunidal 88.00 5.1464 88.00 7.2330 
Tranverse 92.00 6.0641 92.00 9.0598 
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Average 90.00 8.1464 
4. CONCLUSION 

A semi-automated pavement distress characterization analysis using image processing and AI was 
successfully developed based on the results obtained from images acquisition and analysis. The basic tool 
that was used in images capturing process has significantly improved the consistency and quality of photo 
taken at the site. The program development using MATLAB software was success and able to analyse 
cracking images with good consistency and reliability. 

The evaluation of the effectiveness of Artificial Intelligence by detecting of cracking in pavement 
distress was completely achieved by using between kNN and fuzzy KNN classifier. The effectiveness of this 
research includes processing time to classify the pavement distress. On average, this study consumes not 
more than a minute of processing time to detect on cracking of pavement distress. Lower images processing 
time and the capability of repeating the same procedures on a thousand of images without fatigue could be 
the main key in replacing human limited capability in images analysing. The verification of the results 
obtained from AI analysis based on comparison with the field data produced an excellent agreement. 
Thus, the semi-automated AI system was able to identify the cracking pattern from pavement distress based 
on the images analysis using KNN and fuzzy kNN classifier. 
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